{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "# LaB-GATr Detailed Model Reference And Usage Tutorial",
   "id": "530ad1465e98e2a2"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Installation\n",
    "\n",
    "In this notebook we explain the inner blocks of LaB-GATr.\n",
    "\n",
    "Before using this notebook, install the correct dependencies and LaB-GATr as follows:\n",
    "\n",
    "Clone the repo by using\n",
    "```\n",
    "git clone git@github.com:sukjulian/lab-gatr.git\n",
    "```\n",
    "\n",
    "Change the directory into the repository\n",
    "```\n",
    "cd lab-gatr\n",
    "```\n",
    "\n",
    "Optional new Anaconda environment\n",
    "```\n",
    "conda create --name lab-gatr python=3.10\n",
    "conda activate lab-gatr\n",
    "``` \n",
    "Next, install PyTorch and xFormers and other libraries\n",
    "```\n",
    "pip install torch==2.1.0 --index-url https://download.pytorch.org/whl/cu121\n",
    "pip install xformers==0.0.22.post7 --index-url https://download.pytorch.org/whl/cu121\n",
    "pip install torch_geometric==2.4.0\n",
    "pip install torch_scatter torch_cluster -f https://data.pyg.org/whl/torch-2.1.0+cu121.html\n",
    "```\n",
    "Install LaB-GATr itself, which also installs GATr\n",
    "```\n",
    "pip install .\n",
    "```\n",
    "Additionally, if you have made a new Anaconda environment, install Jupyter\n",
    "```\n",
    "pip install jupyter jupyterlab\n",
    "```"
   ],
   "id": "1672d1f3e0fe894"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# Introduction\n",
    "\n",
    "In this tutorial we gradually introduce the building blocks of [LaB-GATr](https://arxiv.org/abs/2403.07536) which is built on top of [GATr](https://www.semanticscholar.org/paper/Geometric-Algebra-Transformer-Brehmer-Haan/4689f6603587e64e87ae36a385e9aab34af2966a). We will introduce the tokenization and the interpolation steps also discussed in our paper and show how to process dummy input with our model from scratch."
   ],
   "id": "c9589d095b2e07ea"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "# Method",
   "id": "b5b3d1f720b37b0c"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## 1. Infrastructure",
   "id": "ff9096e82515eea3"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "from gatr.interface import embed_oriented_plane, extract_translation\n",
    "import torch\n",
    "import gatr\n",
    "from lab_gatr.nn.class_token import class_token_forward_wrapper\n",
    "from xformers.ops.fmha import BlockDiagonalMask\n",
    "\n",
    "from lab_gatr.nn.mlp.geometric_algebra import MLP\n",
    "from lab_gatr.nn.gnn import PointCloudPooling, pool\n",
    "from torch_scatter import scatter\n",
    "from gatr.interface import embed_translation\n",
    "\n",
    "import torch_geometric as pyg\n",
    "from lab_gatr.data import Data\n",
    "from torch_cluster import fps, knn"
   ],
   "id": "5f6cd32689f9f20b",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Let us first create a dummy mesh: n positions and orientations (e.g. surface normal) and an arbitrary scalar feature (e.g. geodesic distance).",
   "id": "fb81be0d48f2dc6d"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "n = 1000\n",
    "\n",
    "pos, orientation = torch.rand((n, 3)), torch.rand((n, 3))\n",
    "scalar_feature = torch.rand(n)"
   ],
   "id": "62fab7f0762fa0fc",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "Next, a point cloud pooling transform for the tokenisation (patching).\n",
    "\n",
    "Nested hierarchy of sub-sampled point clouds. Each coarse-scale point is mapped to a cluster of fine-scale points. Interpolation from the coarse to the fine scales. For correct batching, ```torch_geometric.data.Data.__inc__()``` has to be overridden.\n",
    "\n",
    "The point cloud pooling transform used by LaB-GATr pre-computes information required for coarsening and refinement of the input mesh.\n",
    "\n",
    "For coarsening, we construct a [Voronoi](https://en.wikipedia.org/wiki/Voronoi_diagram)-type diagram. We first sample a fraction of the mesh vertices $ \\mathcal{P} $, approximately equidistantly, via farthest point sampling. Then we create the Voronoi clusters by querying, for each original mesh vertex, the nearest point in the sub-sampled set $ \\mathcal{P} $ via k-nearest neighbours ( $ k = 1 $ ).\n",
    "\n",
    "For interpolation, we simply query for each original mesh vertex the closest $ k $ points in the sub-sampled set $ \\mathcal{P} $. For surface meshes, we choose $ k = 3 $ which corresponds to a triangular reference simplex, while for volume meshes we choose a tetrahedron ( $ k = 4 $ ).\n"
   ],
   "id": "6ce86e22f4b5a9ec"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "class PointCloudPoolingScales():\n",
    "    \"\"\"\n",
    "    Args:\n",
    "        rel_sampling_ratios (tuple): relative ratios for successive farthest point sampling\n",
    "        interp_simplex (str): reference simplex for barycentric interpolation ('triangle' or 'tetrahedron')\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, rel_sampling_ratios: tuple, interp_simplex: str):\n",
    "        self.rel_sampling_ratios = rel_sampling_ratios\n",
    "        self.interp_simplex = interp_simplex\n",
    "\n",
    "        self.dim_interp_simplex = {'triangle': 2, 'tetrahedron': 3}[interp_simplex]\n",
    "\n",
    "    def __call__(self, data: pyg.data.Data) -> Data:\n",
    "\n",
    "        pos = data.pos\n",
    "        batch = data.surface_id.long() if hasattr(data, 'surface_id') else torch.zeros(pos.size(0), dtype=torch.long)\n",
    "\n",
    "        for i, sampling_ratio in enumerate(self.rel_sampling_ratios):\n",
    "\n",
    "            sampling_idcs = fps(pos, batch, ratio=sampling_ratio)  # takes some time but is worth it\n",
    "            # sampling_idcs = torch.arange(0, pos.size(0), 1. / sampling_ratio, dtype=torch.int)\n",
    "\n",
    "            pool_source, pool_target = knn(pos[sampling_idcs], pos, 1, batch[sampling_idcs], batch)\n",
    "            interp_target, interp_source = knn(pos[sampling_idcs], pos, self.dim_interp_simplex + 1, batch[sampling_idcs], batch)\n",
    "\n",
    "            data[f'scale{i}_pool_target'], data[f'scale{i}_pool_source'] = pool_target.int(), pool_source.int()\n",
    "            data[f'scale{i}_interp_target'], data[f'scale{i}_interp_source'] = interp_target.int(), interp_source.int()\n",
    "            data[f'scale{i}_sampling_index'] = sampling_idcs.int()\n",
    "\n",
    "            pos = pos[sampling_idcs]\n",
    "            batch = batch[sampling_idcs]\n",
    "\n",
    "        return Data(**data)\n",
    "\n",
    "    def __repr__(self) -> str:\n",
    "        return f\"{self.__class__.__name__}(rel_sampling_ratios={self.rel_sampling_ratios}, interp_simplex={self.interp_simplex})\""
   ],
   "id": "f3ce6bac0700dde0",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "transform = PointCloudPoolingScales(rel_sampling_ratios=(0.2,), interp_simplex='triangle')\n",
    "dummy_data = transform(pyg.data.Data(pos=pos, orientation=orientation, scalar_feature=scalar_feature))"
   ],
   "id": "b55d9b5347b6865d",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## 2. Building the Model",
   "id": "7abe7e5851297f92"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "A geometric algebra interface to embed your data in $\\mathbf{G}(3, 0, 1)$.\n",
    "\n",
    "\n",
    "\n",
    "Simply giving input and output channels is not enough when working with geometric algebra. Instead, we must define how our data is going to be represented by multivectors (and auxiliary scalars). For convenience, we package this in a `GeometricAlgebraInterface`. Here, we embed the mesh surface normal as planes at each vertex and pass an addtional scalar feature.\n",
    "\n",
    "At the model output, we \"translate\" the multivectors back into Euclidean vectors by interpreting them as translation (pun intended). There are multiple modelling choices for how to perform this extraction (see [paper](https://arxiv.org/abs/2403.07536)).\n"
   ],
   "id": "49f2572dcac22b1a"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "class GeometricAlgebraInterface:\n",
    "    num_input_channels = num_output_channels = 1\n",
    "    num_input_scalars = num_output_scalars = 1\n",
    "\n",
    "    @staticmethod\n",
    "    @torch.no_grad()\n",
    "    def embed(data):\n",
    "\n",
    "        multivectors = embed_oriented_plane(normal=data.orientation, position=data.pos).view(-1, 1, 16)\n",
    "        scalars = data.scalar_feature.view(-1, 1)\n",
    "\n",
    "        return multivectors, scalars\n",
    "\n",
    "    @staticmethod\n",
    "    def dislodge(multivectors, scalars):\n",
    "        return extract_translation(multivectors).squeeze()"
   ],
   "id": "a60f07bad04535db",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### 2.1 Tokenizer\n",
    "\n",
    "The interpolation layer performs a convex combination of multivectors which is compatible with the projective geometric algebra."
   ],
   "id": "2036042a55a7b379"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "def interp(\n",
    "    mlp: torch.nn.Module,\n",
    "    multivectors: torch.Tensor,\n",
    "    multivectors_skip: torch.Tensor,\n",
    "    scalars: torch.Tensor,\n",
    "    scalars_skip: torch.Tensor,\n",
    "    pos_source: torch.Tensor,\n",
    "    pos_target: torch.Tensor,\n",
    "    data: Data,\n",
    "    scale_id: int,\n",
    "    reference_multivector: torch.Tensor\n",
    ") -> torch.Tensor:\n",
    "\n",
    "    pos_diff = pos_source[data[f'scale{scale_id}_interp_source']] - pos_target[data[f'scale{scale_id}_interp_target']]\n",
    "    squared_pos_dist = torch.clamp(torch.sum(pos_diff ** 2, dim=-1), min=1e-16).view(-1, 1, 1)\n",
    "\n",
    "    denominator = scatter(1. / squared_pos_dist, data[f'scale{scale_id}_interp_target'].long(), dim=0, reduce='sum')\n",
    "\n",
    "    multivectors = scatter(\n",
    "        multivectors[data[f'scale{scale_id}_interp_source']] / squared_pos_dist,\n",
    "        data[f'scale{scale_id}_interp_target'].long(),\n",
    "        dim=0,\n",
    "        reduce='sum'\n",
    "    ) / denominator\n",
    "\n",
    "    scalars = scatter(\n",
    "        scalars[data[f'scale{scale_id}_interp_source']] / squared_pos_dist.view(-1, 1),\n",
    "        data[f'scale{scale_id}_interp_target'].long(),\n",
    "        dim=0,\n",
    "        reduce='sum'\n",
    "    ) / denominator.view(-1, 1)\n",
    "\n",
    "    multivectors = torch.cat((multivectors, multivectors_skip), dim=-2)\n",
    "    scalars = torch.cat((scalars, scalars_skip), dim=-1)\n",
    "\n",
    "    return mlp(multivectors, scalars, reference_mv=reference_multivector)"
   ],
   "id": "ce82a1df96835bd0",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "The pooling layer is implemented via PyG message passing. We just have to embed the relative position between points as multivector.",
   "id": "55c6cb4111734917"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "class PointCloudPooling(PointCloudPooling):\n",
    "\n",
    "    def message(\n",
    "        self,\n",
    "        x_j: torch.Tensor,\n",
    "        pos_i: torch.Tensor,\n",
    "        pos_j: torch.Tensor,\n",
    "        scalars_j: torch.Tensor,\n",
    "        reference_multivector_j: torch.Tensor\n",
    "    ) -> torch.Tensor:\n",
    "\n",
    "        multivectors, scalars = self.mlp(\n",
    "            torch.cat((x_j, embed_translation(pos_j - pos_i).unsqueeze(-2)), dim=-2),\n",
    "            scalars=scalars_j,\n",
    "            reference_mv=reference_multivector_j\n",
    "        )\n",
    "\n",
    "        return multivectors, scalars\n",
    "\n",
    "    def aggregate(self, inputs: tuple, index: torch.Tensor, ptr=None, dim_size=None) -> torch.Tensor:\n",
    "        multivectors, scalars = (self.aggr_module(tensor, index, ptr=ptr, dim_size=dim_size, dim=self.node_dim) for tensor in inputs)\n",
    "\n",
    "        return multivectors, scalars"
   ],
   "id": "92927b99c514cad3",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "The complete tokenization module now just wraps pooling and interpolation. For reflection equivariance, we have to keep track of a reference multivector. Furthermore, we support extracting a global class token for classification tasks. This is a lot more efficient than taking the global average of vertex-wise outputs.",
   "id": "ecc84a1da48281e7"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "class Tokeniser(torch.nn.Module):\n",
    "    def __init__(self, geometric_algebra_interface: object, d_model: int, num_latent_channels=None, dropout_probability=None):\n",
    "        super().__init__()\n",
    "        self.geometric_algebra_interface = geometric_algebra_interface()\n",
    "\n",
    "        num_input_channels = self.geometric_algebra_interface.num_input_channels\n",
    "        num_output_channels = self.geometric_algebra_interface.num_output_channels\n",
    "\n",
    "        num_input_scalars = self.geometric_algebra_interface.num_input_scalars\n",
    "        num_output_scalars = self.geometric_algebra_interface.num_output_scalars\n",
    "\n",
    "        num_latent_channels = num_latent_channels or d_model\n",
    "\n",
    "        self.point_cloud_pooling = PointCloudPooling(MLP(\n",
    "            (num_input_channels + 1, num_latent_channels, d_model),\n",
    "            num_input_scalars,\n",
    "            num_output_scalars=num_input_scalars,\n",
    "            plain_last=False,\n",
    "            use_norm_in_first=False,\n",
    "            dropout_probability=dropout_probability\n",
    "        ), node_dim=0)\n",
    "\n",
    "        self.mlp = MLP(\n",
    "            (d_model + num_input_channels, *[num_latent_channels] * 2, num_output_channels),\n",
    "            num_input_scalars=2 * num_input_scalars,\n",
    "            num_output_scalars=num_output_scalars,\n",
    "            use_norm_in_first=False,\n",
    "            dropout_probability=dropout_probability\n",
    "        )\n",
    "\n",
    "        self.cache = None\n",
    "\n",
    "    def forward(self, data: Data) -> torch.Tensor:\n",
    "        multivectors, scalars = self.geometric_algebra_interface.embed(data)\n",
    "\n",
    "        self.cache = {\n",
    "            'multivectors': multivectors,\n",
    "            'scalars': scalars,\n",
    "            'data': data,\n",
    "            'reference_multivector': self.construct_reference_multivector(multivectors, data.batch)\n",
    "        }\n",
    "\n",
    "        (multivectors, scalars), self.cache['pos'] = pool(\n",
    "            self.point_cloud_pooling,\n",
    "            multivectors,\n",
    "            data.pos,\n",
    "            data,\n",
    "            scale_id=0,\n",
    "            scalars=scalars,\n",
    "            reference_multivector=self.cache['reference_multivector']\n",
    "        )\n",
    "\n",
    "        return multivectors, scalars, self.cache['reference_multivector'][data.scale0_sampling_index]\n",
    "\n",
    "    @staticmethod\n",
    "    def construct_reference_multivector(x: torch.Tensor, batch=None):\n",
    "\n",
    "        if batch is None:\n",
    "            reference_multivector = x.mean(dim=(0,1)).expand(x.size(0), 1, -1)\n",
    "\n",
    "        else:\n",
    "            reference_multivector = scatter(x, batch, dim=0, reduce='mean').mean(dim=1, keepdim=True)[batch]\n",
    "\n",
    "        return reference_multivector\n",
    "\n",
    "    def lift(self, multivectors: torch.Tensor, scalars: torch.Tensor) -> torch.Tensor:\n",
    "\n",
    "        if multivectors.size(0) == self.cache['data'].scale0_sampling_index.numel():\n",
    "            multivectors, scalars = interp(\n",
    "                self.mlp,\n",
    "                multivectors,\n",
    "                self.cache['multivectors'],\n",
    "                scalars,\n",
    "                self.cache['scalars'],\n",
    "                self.cache['pos'],\n",
    "                self.cache['data'].pos,\n",
    "                self.cache['data'],\n",
    "                scale_id=0,\n",
    "                reference_multivector=self.cache['reference_multivector']\n",
    "            )\n",
    "\n",
    "        else:\n",
    "            multivectors, scalars = self.extract_class(\n",
    "                self.mlp,\n",
    "                multivectors,\n",
    "                self.cache['multivectors'],\n",
    "                scalars,\n",
    "                self.cache['scalars'],\n",
    "                self.cache['data']\n",
    "            )\n",
    "\n",
    "        return self.geometric_algebra_interface.dislodge(multivectors, scalars)\n",
    "\n",
    "    def extract_class(\n",
    "        self,\n",
    "        mlp: MLP,\n",
    "        multivectors: torch.Tensor,\n",
    "        multivectors_skip: torch.Tensor,\n",
    "        scalars: torch.Tensor,\n",
    "        scalars_skip: torch.Tensor,\n",
    "        data: Data\n",
    "    ) -> torch.Tensor:\n",
    "\n",
    "        if data.batch is None:\n",
    "            multivectors_skip = multivectors_skip.mean(dim=0, keepdim=True)\n",
    "            scalars_skip = scalars_skip.mean(dim=0, keepdim=True)\n",
    "\n",
    "            reference_multivector = self.cache['reference_multivector'][0:1]\n",
    "\n",
    "        else:\n",
    "            multivectors_skip = scatter(multivectors_skip, data.batch, dim=0, reduce='mean')\n",
    "            scalars_skip = scatter(scalars_skip, data.batch, dim=0, reduce='mean')\n",
    "\n",
    "            reference_multivector = self.cache['reference_multivector'][data.ptr[:-1]]\n",
    "\n",
    "        multivectors = torch.cat((multivectors, multivectors_skip), dim=-2)\n",
    "        scalars = torch.cat((scalars, scalars_skip), dim=-1)\n",
    "\n",
    "        return mlp(multivectors, scalars, reference_mv=reference_multivector)\n",
    "\n"
   ],
   "id": "3a333dcb393112e6",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 2.2 LaB-GATr",
   "id": "7e5a71a32ba1fcb9"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Now we can assemble the complete LaB-GATr model. It is comprised of tokenization and a GATr backend. For correct handling of PyG-style batching, we use block-diagonal attention masks. Another highlight is the class token support which we implement via a ```class_token_forward_wrapper```.",
   "id": "c6802731fce0ef60"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "class LaBGATr(torch.nn.Module):\n",
    "\n",
    "    def __init__(\n",
    "        self,\n",
    "        geometric_algebra_interface: object,\n",
    "        d_model: int,\n",
    "        num_blocks: int,\n",
    "        num_attn_heads: int,\n",
    "        num_latent_channels=None,\n",
    "        use_class_token: bool = False,\n",
    "        dropout_probability=None\n",
    "    ):\n",
    "        super().__init__()\n",
    "\n",
    "        num_latent_channels = num_latent_channels or d_model\n",
    "\n",
    "        self.tokeniser = Tokeniser(\n",
    "            geometric_algebra_interface,\n",
    "            d_model,\n",
    "            num_latent_channels=4 * num_latent_channels,\n",
    "            dropout_probability=dropout_probability\n",
    "        )\n",
    "\n",
    "        self.gatr = gatr.GATr(\n",
    "            in_mv_channels=d_model,\n",
    "            out_mv_channels=d_model,\n",
    "            hidden_mv_channels=num_latent_channels,\n",
    "            in_s_channels=geometric_algebra_interface.num_input_scalars,\n",
    "            out_s_channels=geometric_algebra_interface.num_input_scalars,\n",
    "            hidden_s_channels=4 * num_latent_channels,\n",
    "            attention=gatr.SelfAttentionConfig(num_heads=num_attn_heads),\n",
    "            mlp=gatr.MLPConfig(),\n",
    "            num_blocks=num_blocks,\n",
    "            dropout_prob=dropout_probability\n",
    "        )\n",
    "\n",
    "        if use_class_token:\n",
    "            self.gatr.forward = class_token_forward_wrapper(self.gatr.forward)\n",
    "\n",
    "        self.num_parameters = sum(parameter.numel() for parameter in self.parameters() if parameter.requires_grad)\n",
    "        print(f\"LaB-GATr ({self.num_parameters} parameters)\")\n",
    "\n",
    "    def forward(self, data: Data) -> torch.Tensor:\n",
    "        multivectors, scalars, reference_multivector = self.tokeniser(data)\n",
    "\n",
    "        multivectors, scalars = self.gatr(\n",
    "            multivectors,\n",
    "            scalars=scalars,\n",
    "            attention_mask=self.get_attn_mask(data),\n",
    "            join_reference=reference_multivector\n",
    "        )\n",
    "\n",
    "        return self.tokeniser.lift(multivectors, scalars)\n",
    "\n",
    "    @staticmethod\n",
    "    def get_attn_mask(data: Data):\n",
    "\n",
    "        if data.batch is None:\n",
    "            attn_mask = None\n",
    "\n",
    "        else:\n",
    "            batch = data.batch[data.scale0_sampling_index]\n",
    "            attn_mask = BlockDiagonalMask.from_seqlens(torch.bincount(batch).tolist())\n",
    "\n",
    "        return attn_mask"
   ],
   "id": "c9f7b149dd1d85f",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## 3. Usage",
   "id": "33520970d4afc701"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "\n",
    "print(f\"Running test on {device}!\")\n",
    "\n",
    "model = LaBGATr(GeometricAlgebraInterface, d_model=8, num_blocks=10, num_attn_heads=4, use_class_token=False).to(device)"
   ],
   "id": "218c6f4f2be5fd89",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Generate some output with the dummy data to verify that the model functions. Training or inference from here on is the same as any PyTorch model.\n",
   "id": "854cf313b105b230"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "output = model(dummy_data.to(device))\n",
    "\n",
    "print(output.shape)"
   ],
   "id": "9d023f910c1cccdc",
   "outputs": [],
   "execution_count": null
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
